Image processing system, image processing method, and non-transitory storage medium storing image processing program

ABSTRACT

An image processing system includes at least one image processing apparatus, an information terminal, and an information processing device configured to communicate with the at least one image processing apparatus and the information terminal. The information terminal includes a specifying information transmitter configured to transmit, to the information processing device, specifying information for specifying, from the at least one image processing apparatus, a first image processing apparatus for inputting and a second image processing apparatus for outputting. The information processing device includes an input data retrieval unit configured to obtain input data from the first image processing apparatus for inputting based on the specifying information received from the information terminal, and an output data transmitter configured to transmit, based on the received specifying information, output data corresponding to the obtained input data to the second image processing apparatus for outputting.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application filed under 35 U.S.C.111(a) claiming the benefit under 35 U.S.C. 120 of U.S. patentapplication Ser. No. 15/068,856 filed on Mar. 14, 2016, which is basedupon and claims the benefit of priority to Japanese Priority PatentApplication No. 2015-055965 filed on Mar. 19, 2015, the entire contentsof which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing system, an imageprocessing method, and a non-transitory storage medium storing an imageprocessing program.

2. Description of the Related Art

An image forming apparatus having a plurality of functions has beenknown, such as a multifunction peripheral/product (MFP) having, forexample, a printing function, a copying function, a scanning function,and a facsimile transmission function. Additionally, a technique hasbeen known such that an image forming function of an image formingapparatus is operated by using a smart device, such as a smartphone anda tablet terminal, by using an application program interface (API)provided by the image forming apparatus.

Furthermore, an output sorting device has been known that determines,based on user specific information extracted from an output request froma terminal, an image forming apparatus as an output destination, anddrawing data corresponding to the output request is output to thedetermined output destination (c.f. Patent Document 1 (JapaneseUnexamined Patent Publication No. 2013-25774, for example)).

According to related art, it is possible to operate a function of animage forming apparatus (e.g., a MFP) by using an information terminal;and the information terminal can access, for example, print data andscan data, which are to be processed. Accordingly, in an environment,such as a Bring Your Own Device (BYOD) environment where a personalsmart device is connected to an office network and used, information mayleak because data to be processed is stored in a personal smart device,for example.

For a case of controlling, by an information terminal, image processingthat is to be executed by a plurality of image processing devices, suchas a case where image information that is written on an interactivewhiteboard is to be printed by a MFP, it has been difficult to controlthe process without storing, in the information terminal, data to beprocessed.

In this manner, according to the related art, it has been difficult tocontrol, by using an information terminal, such as a smart device, imageprocessing that is to be executed by a plurality of image processingdevices, while preventing the data to be processed from being leaked.

There is a need for an image processing system that makes it easier tocontrol, by using an information terminal, such as a smart device, imageprocessing that is to be executed by a plurality of image processingdevices, while preventing information of the data to be processed frombeing leaked.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, there is provided animage processing system including at least one image processingapparatus, an information terminal, and an information processing deviceconfigured to communicate with the at least one image processingapparatus and the information terminal, wherein the information terminalincludes a specifying information transmitter configured to transmit, tothe information processing device, specifying information forspecifying, from the at least one image processing apparatus, an imageprocessing apparatus for inputting and an image processing apparatus foroutputting, and wherein the information processing device includes aninput data retrieval unit configured to obtain input data from the imageprocessing apparatus for inputting based on the specifying informationreceived from the information terminal, and an output data transmitterconfigured to transmit, based on the received specifying information,output data that is based on the obtained input data to the imageprocessing apparatus for outputting.

According to another aspect of the present invention, there is provideda non-transitory storage medium storing an image processing program,wherein an image processing system comprises at least one imageprocessing apparatus, an information terminal, and an informationprocessing device configured to communicate with the at least one imageprocessing apparatus and the information terminal to function, and theimage processing program causes the information processing device tofunction as a specifying information receiver configured to receivespecifying information for specifying, from the at least one imageprocessing apparatus, a first image processing apparatus for inputtingand a second image processing apparatus for outputting; an input dataretrieval unit configured to obtain input data from the first imageprocessing apparatus for inputting, based on the received specifyinginformation; and an output data transmitter configured to transmit,based on the received specifying information, output data correspondingto the obtained input data to the second image processing apparatus foroutputting.

According to another aspect of the present invention, there is provideda method to be executed by an image processing device, wherein an imageprocessing system includes at least one image processing apparatus, aninformation terminal, and the information processing device configuredto communicate with the at least one image processing apparatus and theinformation terminal to function. The method includes receivingspecifying information for specifying, from the at least one imageprocessing apparatus, a first image processing apparatus for inputtingand a second image processing apparatus for outputting; obtaining inputdata from the first image processing apparatus for inputting, based onthe received specifying information; and transmitting, based on thereceived specifying information, output data corresponding to theobtained input data to the second image processing apparatus foroutputting.

According to the embodiment of the present invention, an imageprocessing system can be provided that makes it easier to control, byusing an information terminal, such as a smart device, image processingthat is to be executed by a plurality of image processing devices, whilepreventing information of the data to be processed from being leaked.

Other objects, features and advantages of the present invention willbecome more apparent from the following detailed description when readin conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of a configuration of animage processing system according to an embodiment;

FIG. 2 is a diagram illustrating another example of the configuration ofthe image processing system according to the embodiment;

FIG. 3 is a diagram illustrating an example of a hardware configurationof a relay server according to the embodiment;

FIG. 4 is a diagram illustrating an example of a hardware configurationof an information terminal according to the embodiment;

FIG. 5 is a diagram illustrating an example of a hardware configurationof an image processing apparatus according to the embodiment;

FIG. 6 is a functional configuration diagram of an example of the imageprocessing system according to the embodiment;

FIG. 7 is a sequence chart illustrating an example of image processingaccording to the embodiment;

FIG. 8 is a diagram illustrating an example of a device identificationscreen according to the embodiment;

FIG. 9 is a diagram illustrating an example of device informationaccording to the embodiment;

FIG. 10 is a diagram illustrating an example of a setting screenaccording to the embodiment;

FIG. 11 is a diagram illustrating an example of specifying informationaccording to the embodiment;

FIG. 12 is a diagram illustrating an example of a convertible listaccording to the embodiment;

FIG. 13 is a flowchart illustrating an example of a determinationprocess and a conversion process of a data type according to theembodiment;

FIG. 14A is a diagram illustrating an example of an input data retrievalrequest according to the embodiment;

FIG. 14B is a diagram illustrating an example of an output dataretrieval request according to the embodiment;

FIG. 15 is a diagram illustrating an example of a functionalconfiguration of the image processing system according to anotherembodiment;

FIG. 16 is a sequence chart illustrating an example of the imageprocessing according to the other embodiment;

FIG. 17A is a diagram illustrating an example of specifying informationaccording to the other embodiment;

FIG. 17B is a diagram illustrating an example of identifying informationfor identifying a process;

FIG. 18 is a diagram illustrating another example of the specifyinginformation according to the other embodiment;

FIG. 19 is a diagram illustrating an example of the functionalconfiguration of the image processing system according to a furtherembodiment;

FIG. 20 is a sequence chart illustrating an example of a registrationprocess according to the further embodiment;

FIG. 21 is a sequence chart illustrating an example of the imageprocessing according to the further embodiment;

FIG. 22 is a diagram showing an example of list information according tothe further embodiment;

FIG. 23 is a diagram illustrating an example of a selection screen of aninput/output device according to the further embodiment; and

FIG. 24 is a diagram illustrating an example of the specifyinginformation according to the further embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention are described below by referring tothe accompanying drawings.

<System Configuration>

FIG. 1 is a diagram illustrating an example of a configuration of animage processing system 100 according to an embodiment. The imageprocessing system 100 may include, for example, a relay server 101; anaccess point 103; an electronic whiteboard 104-1; a MultifunctionPeripheral/Product (MFP) 104-2; a printer 104-3; and a projector 104-4,which are connected to a network 105, such as the Internet or a localarea network (LAN). Additionally, the image processing system 100 mayinclude an information terminal 102 that can be connected to the network105 through the access point 103.

Here, the electronic whiteboard 104-1, the MFP 104-2, the printer 104-3,and the projector 104-4 are an example of at least one image processingapparatus that is included in the image processing system 100. The atleast one image processing apparatus may include another imageprocessing apparatus, such as a scanner, a camera, a display, a signage,a television, and a video conference device.

In the following description, any image processing apparatus of the atleast one image processing apparatus may be denoted as an “imageprocessing apparatus 104.”

The relay server 101 is an information processing device that controls,in response to a command from the information terminal 102, a process ofinputting image data from the at least one image processing apparatus104, and a process of outputting image data to the at least one imageprocessing apparatus 104. The relay server 101 controls, for example, aprocess of retrieving image data from an image processing apparatus 104for inputting (e.g., the MFP 104-2), which is specified by theinformation terminal 102; and a process of outputting image data to animage processing apparatus 104 for outputting (e.g., the electronicwhiteboard 104-1), which is specified by the information terminal 102.

The information terminal 102 is an information processing device, suchas a smartphone, a tablet terminal, and a personal computer (PC). Theinformation terminal 102 instructs the relay server 101 to execute aprocess of inputting image data from the above-described at least oneimage processing apparatus 104; and a process of outputting image datato the at least one image processing apparatus 104.

The access point 103 is a connection device for connecting a wirelessLAN device, such as the information terminal 102, to the network 105.The access point 103 executes an authentication process for a wirelessLAN device that requests to connect to the network 105; and, in responseto detecting that the authentication is approved for the wireless LANdevice, the access point 103 allows the wireless LAN device to connectto the network 105. The wireless LAN device for which the authenticationis approved by the access point 103 is allowed to execute datacommunication with a network device, such as the relay server 101 andthe at least one image processing apparatus 104, which are connected tothe network 105.

The electronic whiteboard 104-1 is an image processing apparatus, whichmay be formed by installing a touch panel on a flat panel display, suchas a liquid crystal display, or on a projection screen of a projector. Auser can input drawing data on the electronic whiteboard 104-1 by handwriting, while the electronic whiteboard 104-1 is displaying a material,for example.

The MFP 104-2 is an image processing apparatus with a plurality of imageforming functions, such as a printing function, a copying function, ascanning function, and a facsimile function. A printer 104-3 is an imageprocessing apparatus with an image printing function. The projector104-4 is an image processing apparatus with an image projectionfunction.

With the above-described configuration, a user transmits, to the relayserver, the specifying information for specifying the image processingapparatus 104 for inputting and the image processing apparatus 104 foroutputting by using the information terminal 102. Further, uponreceiving the specifying information from the information terminal 102,the relay server 101 obtains input data from the image processingapparatus 104 for inputting based on the specifying information, and therelay server 101 transmits, to the image processing apparatus 104 foroutputting, output data based on the obtained input data. At this time,the information terminal 102 can control image processing by theplurality of image processing devices 104, without obtaining, by theinformation terminal 102, the data to which the image processing isapplied.

In this manner, according to the embodiment, the image processing system100 can be provided that makes it easier to control, by using theinformation terminal 102, such as a smart device, image processing thatis to be executed by the plurality of image processing devices 104,while preventing information of the data to be processed from beingleaked.

Note that the system configuration of FIG. 1 is for exemplifyingpurposes only, and various system configurations can be applied for theimage processing system 100 according to the embodiment.

FIG. 2 is a diagram showing another example of a system configuration ofthe image processing system 100 according to the embodiment. In theexample of FIG. 2, the electronic whiteboard 104-1, the MFP 104-2, theprinter 104-3, and the projector 104-4 are connected to the network 105through the access point 103. In this manner, the at least one imageprocessing apparatus 104 may be connected to the network 105 by radiocommunication, such as the radio communication through a wireless LAN.

Additionally, the image processing system 100 may include the imageprocessing apparatuses 104 that are connected by a wired LAN; and theimage processing apparatuses 104 that are connected by a wireless LAN.

<Hardware Configuration>

(Hardware Configuration of the Relay Server)

FIG. 3 is a diagram illustrating an example of a hardware configurationof the relay server 101 according to the embodiment. The relay server101 may be configured as a general-purpose computer. The relay server101 may include, for example, a central processing unit (CPU) 301; arandom access memory (RAM) 302; a read-only memory (ROM) 303; a storageunit 304; a network interface (I/F) 305; an external I/F 306; a displayunit 307; an input unit 308; and a bus 309.

The CPU 301 is a processor for implementing functions of the relayserver 101 by reading out one or more programs and data stored in theROM 303 and/or the storage unit 304, writing the read one or moreprograms and data in the RAM 302, and executing one or more processes.The RAM 302 is a volatile memory that is used as a work area of the CPU301.

The ROM 303 is a non-volatile memory for maintaining one or moreprograms and data, even if a power supply of the relay server 101 isturned off. The ROM 303 may be formed of a flash ROM, for example. Thestorage unit 304 is a storage device, such as a hard disk drive (HDD)and a solid state drive (SSD), for example. The storage unit 304 stores,for example, an operating system (OS), one or more application programs,and various types of data.

The network I/F unit 305 is a communication interface of, for example, awired LAN and/or a wireless LAN for connecting the relay server 101 tothe network 105; and for transmitting data to and receiving data fromthe information terminal 102 or the image processing apparatus 104.

The external I/F unit 306 is an interface for connecting the relayserver 101 to an external device. Examples of the external deviceincludes a recording medium, such as a USB memory, a memory card, and anoptical disk; and various types of electronic devices.

The display unit 307 is a display device, such as a liquid crystaldisplay (LCD), for displaying a processing result of the relay server101, for example. The input unit 308 is an input device, such as akeyboard, a mouse, and a touch panel, for receiving an input foroperating the relay server 101. Note that the display unit 307 or theinput unit 308 may be formed outside the relay server 101. The bus 309is connected to the above-described components of the relay server 101.The bus 309 transmits an address signal, a data signal, and varioustypes of control signals.

(Hardware Configuration of the Information Terminal)

FIG. 4 is a diagram illustrating an example of a hardware configurationof the information terminal 102 according to the embodiment. Theinformation terminal 102 may be configured as a general-purposecomputer. The information terminal 102 may include, for example, a CPU401; a RAM 402; a ROM 403; a storage unit 404; a camera unit 405; a nearfield communication unit (NFC) 406; a Bluetooth (registered trademark)Low Energy (which is referred to as the BLE, hereinafter) communicationunit 407; a network I/F unit 408; a display input unit 409; and a bus410.

Here, the configurations of the CPU 401, the RAM 402; the ROM 403; thestorage unit 404; and the bus 410 are the same as the configurations ofthe corresponding components of the relay server 101 of FIG. 3. Thus,the differences of the configuration of the information terminal 102from the configuration of the relay server 101 are mainly described.

The camera unit 405 is an image capturing device for capturing an image.For example, the camera unit 405 is used for capturing an image of atwo-dimensional code of the image processing apparatus 104, such as aquick response (QR) code (registered trademark), to obtain addressinformation and/or device information of the image processing apparatus104.

The NFC communication unit 406 is a communication device forcommunicating with a NFC reader/writer based on the NFC communication,upon detecting, for example, that a distance between the NFCcommunication unit 406 of the information terminal 102 and the NFCreader/writer of the image processing apparatus 104 is less than orequal to a predetermined distance (e.g., less than or equal to 10 cm).The NFC communication unit 406 is used for obtaining, from the NFCreader/writer of the image processing apparatus 104, address informationand/or device information of the image processing apparatus 104, forexample.

The BLE communication unit 407 is a communication device for executing,for example, BLE-based communication with the image forming apparatus104. The BLE is one of extended standards of the Bluetooth standard,which is a short range radio communication standard. The BLE isdeveloped as a part of the Bluetooth 4.0 standard, and the BLE specifiesa short range radio communication technology. According to the BLE-basedcommunication, data communication by short range radio communication canbe implemented with energy consumption that is lower than the energyconsumption by normal Bluetooth communication.

Note that, the NFC communication unit 406 and the BLE communication unit407 are an example of the short range radio communication unit that isincluded in the information terminal 102. The information terminal 102may execute communication with the image processing apparatus 104 byanother short range radio communication scheme, instead of using the NFCcommunication unit 406 and the BLE communication unit 407.

The network I/F unit 408 is a communication interface, such as acommunication interface for a wireless LAN, for connecting theinformation terminal 102 to the network 105, and for transmitting datato and receiving data from the relay server 101.

The display input unit 409 is a device for displaying and inputting,such as a touch panel display where a touch panel and a display areintegrated. The display input unit 409 may include an input unit foroperating the information terminal 102; and a display unit fordisplaying a processing result of the information terminal 102. Here,the display input unit 409 may include a display unit and an input unit,which are separate units.

(Hardware Configuration of the Image Processing Apparatus)

FIG. 5 is a diagram illustrating an example of a hardware configurationof the image processing apparatus 104 according to the embodiment. Here,an example of a hardware configuration of a MFP is described as anexample of the hardware configuration of the image processing apparatus104.

The image processing apparatus (MFP) 104 may include a controller 501;an image reading unit 502; a plotter unit 503; an imageprocessing/engine controller 504; a FAX unit 505; a storage unit 506; anoperation unit 507; and a NFC Reader/Writer (R/W) unit 508.

The controller 501 may be configured as a general-purpose computer. Thecontroller 501 may include, for example, a CPU 509; a RAM 510; a ROM511; an external I/F unit 512; an operation I/F unit 513; a NFC I/F unit514; a BLE communication unit 515; and a network I/F unit 516, which areconnected mutually through a bus 517.

The CPU 509 is a processor for implementing functions of the imageprocessing apparatus 104 by reading out one or more programs and datastored in the ROM 511 and/or the storage unit 506, writing the read oneor more programs and data in the RAM 510, and executing one or moreprocesses. The RAM 510 is a volatile memory that is used as a work areaof the CPU 509. The ROM 511 is a non-volatile memory for maintaining oneor more programs and data, even if a power supply of the imageprocessing apparatus 104 is turned off. The ROM 511 may be formed of aflash ROM, for example.

The external I/F unit 512 is an interface with an external device.Examples of the external device includes a recording medium, such as aUSB memory, a memory card, and an optical disk; and various types ofelectronic devices. The operation I/F unit 513 is an interface forconnecting the operation unit 507 to the controller 501. The NFC I/Funit 514 is an interface for connecting the NFC R/W unit 508 to thecontroller 501. Here, the NFC R/W unit 508 is for reading and writingdata for an NFC device through the NFC communication.

The BLE communication unit 515 is a radio communication unit forexecuting short range radio communication based on the BLE. The networkI/F unit 516 is a communication interface for connecting the imageprocessing apparatus 104 to the network 105, and for transmitting datato and receiving data from the relay server 101 that is connected to thenetwork 105. The above-described components are mutually connectedthrough the bus 517. The bus 517 may transmit an address signal, a datasignal, and various types of control signals, for example.

The image reading unit 502 may be, for example, a scanner engine forreading an image of a document, in accordance with the control of theimage processing/engine controller 504. The plotter unit 503 may be, forexample, a printer engine for outputting an image on a paper sheet, inaccordance with the control of the image processing/engine controller504. The image processing/engine controller 504 executes imageprocessing by controlling the image reading unit 502 and the plotterunit 503.

The FAX unit 505 may include, for example, a hardware engine forexecuting facsimile transmission and reception, and a controller of thehardware engine. The storage unit 506 is a storage device, such as a HDDand a SSD. The storage device 506 may store various types of informationand data, such as an OS, an application, and image data.

The operation unit 507 is an input unit for receiving an input operationfrom a user, and at the same time, the operation unit 507 is a displayunit for displaying information for the user. Note that, in the exampleof FIG. 4, the operation unit 507 includes the NFC R/W unit 508 forreading and writing data for the NFC device by the NFC communication.The configuration of FIG. 5 is for exemplifying purposes only. The NFCR/W unit 508 may be provided separately from the operation unit.

As the MFP 104 depicted in FIG. 5, the image processing apparatus 104according to the embodiment may include, for example, a configuration asa computer; and an image processing engine for implementing an imageprocessing function of the image processing apparatus 104.

<Functional Configuration>

FIG. 6 is a diagram illustrating a functional configuration of the imageprocessing system 100 according to the embodiment. The image processingsystem 100 may include, as examples of the relay server 101, theinformation terminal 102, and the at least one image processingapparatus 104, the electronic whiteboard 104-1; the MFP 104-2; theprinter 104-3; and the projector 104-4, for example. Here, the depictionof the access point 103 is omitted because the access point 103 does notaffect the functional configuration of the embodiment.

(Functional Configuration of the Relay Server)

The relay server 101 may include a communication unit 601; a specifyinginformation receiver 602; a data type determination unit 603; an inputdata retrieval unit 604; a converter 605; an output data transmitter606; and a storage unit 607.

The communication unit 601 is a unit for connecting the relay server 101to the network 105, and for implementing data communication with theinformation terminal 102 and the at lest one image processing apparatus104. The communication unit 601 may be implemented by the network I/Funit 305 of FIG. 3, for example.

The specifying information receiver 602 receives, from the informationterminal 102, specifying information for specifying, from the at leastone image processing apparatus 104, the image processing apparatus 104for inputting and the image processing apparatus 104 for outputting. Thespecifying information receiver 602 may preferably store the receivedspecifying information in the storage unit 607 as the specifyinginformation 608.

The data type determination unit 603 determines a data type of the inputdata obtained from the image processing apparatus 104 for inputting,based on the information on the data type that can be used for the imageprocessing apparatus 104 for inputting, and the information on the datatype that can be used for the image processing apparatus 104 foroutputting, which are included in the specifying information 608. Thespecific determination method is described below.

The input data retrieval unit 604 obtains the input data from the imageprocessing apparatus 104 for inputting, based on the specifyinginformation received, by the specifying information receiver 602, fromthe information terminal 102. For example, the input data receiver 604obtains, from the image processing apparatus 104 for inputting that isspecified by the specifying information, the input data in the data typedetermined by the data type determination unit 603. The input dataretrieval unit 604 may preferably store the obtained input data in thestorage unit 607 as content data 609.

The converter 605 converts, based on the information on the data typethat can be processed by the image processing apparatus 104 foroutputting, which is included in the specifying information 608, thecontent data (the input data) 609 obtained by the input data retrievalunit 604 into data in the data type that can be processed by the imageprocessing apparatus 104 for outputting. Note that, if the content data609 obtained by the input data retrieval unit 604 can be processed bythe image processing apparatus 104 for outputting as it is, theconverter 605 may not convert the data type of the content data 609.

The output data transmitter 606 outputs, based on the specifyinginformation 608 received by the specifying information receiver 602,output data based on the input data obtained by the input data retrievalunit 604 to the image processing apparatus 104 for outputting. Forexample, if the image processing apparatus 104 for outputting canprocess the data type of the input data obtained by the input dataretrieval unit 604, the output data transmitter 606 transmits the inputdata obtained by the input data retrieval unit 604 to the imageprocessing apparatus 104 for outputting. Further, if the imageprocessing apparatus 104 for outputting may not process the data type ofthe input data, the output data transmitter 606 transmits, to the imageprocessing apparatus 104 for outputting, the image data that isconverted, by the converter 605, into the image data in the data typethat can be processed by the image processing apparatus 104 foroutputting.

Note that the above-described specifying information receiver 602, thedata type determination unit 603, the input data retrieval unit 604, theconverter 605, and the output data transmitter 606 can be implemented byan image processing program executed by the CPU 301 of FIG. 3, forexample.

The storage unit 607 may be, for example, a device for storing varioustypes of information and data, such as the specifying information 608that is received by the specifying information receiver 602, and thecontent data 609 that is obtained by the input data retrieval unit 604.The storage unit 607 can be implemented, for example, by a storagedevice, such as the RAM 302 and/or the storage unit 304 of FIG. 3, andone or more programs executed by the CPU 301.

(Functional Configuration of the Information Terminal)

The information terminal 102 may include a communication unit 610; adevice identification unit 611; a device information retrieval unit 612;a specifying information transmitter 613; a setting unit 614; a shortrange radio communication unit 615; an image capture unit 616; a displayinput unit 617; and a storage unit 618.

The communication unit 601 is for connecting the information terminal102 to the network 105, and for executing data communication with therelay server 101 and the at least one image processing apparatus 601.The communication unit 601 may be implemented, for example, by thenetwork I/F unit 408 of FIG. 4.

The device identification unit 611 is for identifying, from the at leastone image processing apparatus 104, the image processing apparatus 104for inputting and the image processing apparatus 104 for outputting,based on a user's operation, for example. The device identification unit611 can cause the display unit 617 to display a device identificationscreen. The device identification unit 611 identifies, by using, forexample, the information received by the display input unit 617, theinformation obtained by the short range radio communication unit 615, orthe information captured by the image capture unit 617, the imageprocessing apparatus 104 for inputting and the image processingapparatus 104 for outputting.

The device information retrieval unit 612 obtains, from the imageprocessing apparatus 104 for inputting that is identified by the deviceidentification unit 611, the device information (capabilityinformation). Further, the device information retrieval unit 612 obtainsthe device information from the image processing apparatus 104 foroutputting that is identified by the device identification unit 611. Thedevice information is described below.

The specifying information transmitter 613 transmits, to the relayserver 101, the specifying information that is identified by the deviceidentification unit 611 and that specifies, from the at least one imageprocessing apparatus 104, the image processing apparatus 104 forinputting and the image processing apparatus 104 for outputting. Thespecifying information includes address information, such as an IPaddress, a host name, or a MAC address of the image processing apparatus104 for inputting, and an IP address, a host name, or a MAC address ofthe image processing apparatus 104 for outputting. In the embodiment,the specifying information includes the device information of the imageprocessing apparatus 104 for inputting and the device information of theimage processing apparatus 104 for outputting, which are obtained by thedevice information retrieval unit 612.

In response to detecting that at least one of the device information ofthe image processing apparatus 104 for inputting and the deviceinformation of the image processing apparatus 104 for outputtingincludes setting information for setting the device, the setting unit614 receives an input for changing the setting information from a userthrough the display input unit 617.

Here, the device identification unit 611, the device informationretrieval unit 612, the specifying information transmitter 613, and thesetting unit 614 may be implemented by one or more programs executed bythe CPU 401 of FIG. 4, for example.

The short range radio communication unit 615 is for executing shortrange radio communication, such as the NFC communication and the BLEcommunication, with the image processing apparatus 104. The short rangeradio communication unit 615 can be implemented by the NFC communicationunit 406 and/or the BLE communication unit 407, for example.

The image capture unit 616 is for capturing image of a bar code or atwo-dimensional code corresponding to the image processing apparatus104. The image capture unit 616 can be implemented by the camera unit405, for example.

The display input unit 617 is for displaying the device identificationscreen and the setting screen, and at the same time, for receiving auser's operation, for example. The display input unit 617 can beimplemented, for example, by the display input unit 409 of FIG. 4 andone or more programs executed by the CPU 401.

The storage unit 618 is for storing various types of information anddata including, for example, device information of the image processingapparatus 104 obtained by the device information retrieval unit 612. Thestorage unit 618 can be implemented, for example, by the storage device,such as the RAM 402 and the storage unit 404 of FIG. 4, and one or moreprograms executed by the CPU 401.

<Flow of the Process>

(Flow of the Image Processing)

FIG. 7 is a sequence chart illustrating an example of image processingaccording to the embodiment. Note that, in FIG. 7, the arrows indicatedby the dashed lines indicate operations on the information terminal 102by the user.

At step S701, upon detecting an operation, by the user, to start anapplication, which is installed in the information terminal 102, for theimage processing system 100 (an image processing program), the imageprocessing system 100 starts image processing.

At step S702, upon receiving the operation to start the application, theinformation terminal 102 causes the display input unit 617 to displaythe device identification screen for prompting the user to specify theimage processing apparatus 104 for inputting (which is referred to asthe input device, hereinafter) and the image processing apparatus 104for outputting (which is referred to as the output device, hereinafter).

At step S703, a user performs an operation to specify the input device(specifying operation), in accordance with the device identificationscreen displayed on the display input unit 617 of the informationterminal 102.

FIG. 8 is a diagram illustrating an example of a device identificationscreen 801 according to the embodiment. At step S702 of FIG. 7, theinformation terminal 102 causes the display input unit 617 to displaythe device identification screen 801, which is illustrated in FIG. 8,for example. The device identification screen 801 includes, for example,a select input device button 802 for selecting the input device, aselect output device button 803 for selecting the output device, and arun button 804.

In response to detecting that the select input button 802 or the selectoutput button 803 is pressed (tapped) at the device identificationscreen 801, a selection method specifying screen 805 depicted in FIG. 8is displayed on the display input unit 617, for example. The selectionmethod specifying screen 805 may include, for example, a directconnection button 806, a connection by touching button 807, and aconnection by two-dimensional code button 808. The user can specify themethod of selecting the input device or the output device in theselection method specifying screen 805.

For example, upon detecting that the direct connection button 806 ispressed in the selection method specifying screen 805 of FIG. 8, theinformation terminal 102 causes the display input unit 617 to display adestination setting screen 809, such as the screen depicted in FIG. 8.In the destination setting screen 809 of FIG. 8, the user inputs, to aninput screen 810, the IP address, the host name, or the MAC address ofthe image processing apparatus 104 that is desired to be used as theinput device (or the output device), and the user presses a connectionbutton 811. In this manner, the user can specify the image processingapparatus 104 that is used as the input device (or the output device).

In response to detecting that the connection by touching button 807 ispressed in the selection method specifying screen 805 of FIG. 8, theinformation terminal 102 causes the display input unit 617 to display amessage screen 812 as depicted in FIG. 8. During the time period inwhich the message screen 812 is displayed, the information terminal 102transitions to a mode were a radio wave of the NFC communication, forexample, is detected by the short range radio communication unit 615.Upon detecting that the information terminal 102 approaches thespecified image processing apparatus 104, the information terminal 102causes the short range radio communication unit 615 to obtain, from theimage processing apparatus 104, address information (networkinformation), such as the IP address, the host name, or the MAC addressof the image processing apparatus 104.

Note that, at this time, the short range radio communication unit 615may obtain, in addition to the address information of the imageprocessing apparatus 104, the device information, which is describedbelow. Additionally, the information terminal 102 may obtain informationfrom the image processing apparatus 104 by using a short range radiocommunication scheme other than the NFC-based communication scheme, suchas the BLE-based communication scheme.

Further, upon detecting that the connection by two-dimensional codebutton 808 is pressed in the selection method specifying screen 805 ofFIG. 8, the information terminal 102 causes the display input unit 617to display a two-dimensional code reading screen 813, such as the screenshown in FIG. 8, for example. In response to detecting that, in thisstate, an image of the two-dimensional code is captured by the imagecapture unit 616, the device identification unit 611 obtains thedestination information of the image processing apparatus 104, which isembedded in the two-dimensional code. Here, in the two-dimensional code,not only the destination information, but also the device information,which is described below, may be embedded.

Here, referring back to FIG. 7, the description of the sequence chart iscontinued.

At step S704, the information terminal 102 transmits a request to obtaindevice information to the input device that is identified (specified) atstep S703 (e.g., the electronic whiteboard 104-1).

At step S705, the information terminal 102 obtains the deviceinformation transmitted from the input device to which the request toobtain the device information is transmitted (e.g., from the electronicwhiteboard 104-1).

At step S706, the user performs an operation to identify the outputdevice in the device identification screen 801, which is displayed onthe display input unit 617 of the information terminal 102. Here, theoperation to identify the output device is the same as theabove-described operation to identify the input device.

At step S707, the information terminal 102 transmits a request to obtainthe device information to the output device that is identified(specified) at step S706 (e.g., the MFP 104-2).

At step S708, the information terminal 102 obtains the deviceinformation from the output device to which the request to obtain thedevice information is transmitted (e.g., from the MFP 104-2).

FIG. 9 is a diagram illustrating an example of the device informationaccording to the embodiment. The device information (the capabilityinformation) 901 may include information, such as “type,” “input,” and“output.”

The “type” information indicates a type of the image processingapparatus 104, such as the electronic whiteboard (whiteboard), the MFP(mfp), and the projector (projector).

The “input” information indicates a specification for inputting acontent (input data). The device information of the device that does notsupport the input may not include the “input” information. Similarly,the “output” information indicates a specification for outputting acontent (output data). The device information of the device that doesnot support the output may not include the “output” information.

Additionally, the “input” information and the “output” information mayinclude “mime_type” information and “settings” information, for example.

The “mime_type” information is list information of types of a content(data types) that can be processed by the image processing apparatus104. The “settings” information indicates available setting parameters,and ranges in which the setting parameters can be set.

In the example of the device information 901 that is illustrated in FIG.9, it is described that the image processing apparatus 104 is a MFP, andthat PDF (application/pdf) or JPEG (image/jpeg) can be used as the datatype of the input (scan) image. Additionally, it is described that acolor mode (color_mode), a resolution (resolution) and a surface of adocument (2 sided) can be specified as the setting of the imageprocessing apparatus 104 during the input.

Additionally, it is described that the image processing apparatus 104can use PDF (application/pdf) and Postscript (application/postscript) asoutput (print) image data types. Furthermore, it is described that acolor mode (color_mode), a resolution (resolution), a side of a document(2 sided), double-sided printing (nup), and a number of copies (copies)can be specified as the setting of the image processing apparatus 104during the output.

Referring back to FIG. 7, the description of the sequence chart iscontinued.

At step S709 of FIG. 7, the user performs an execution operation torequest execution of image processing from the information terminal 102,for example, by pressing the run button 804 included in the deviceidentification screen 801 of the display input unit 617 of theinformation terminal 102.

Note that, if it is determined that the “settings” information isincluded in the device information that is obtained from the inputdevice and the output device, the setting screen can be displayed in theinformation terminal 102, prior to the execution operation at step S709,to change the setting parameters.

FIG. 10 is a diagram illustrating an example of the setting screen 1001according to the embodiment. In the example of FIG. 10, the settingscreen 1001 may include a setting field 1002 for setting “color mode,” asetting field 1003 for setting “resolution,” a setting field 1004 forsetting “surface of document,” a setting field 1005 for setting“layout,” and a setting field 1006 for setting “number of copies.” Theuser can change the setting value by tapping the corresponding settingfield.

For example, upon detecting that the setting field 1002 for the “colormode” in the setting screen 1001 is tapped by the user, a setting screen1007 for setting the color mode is displayed on the display input unit617. In the setting screen 1007 for the color mode, for example, asetting button 1008 for setting the “color,” and a setting button 1009for setting the “monochrome” are displayed, so that the user can selectthe color mode by tapping one of the setting button 1008 and the settingbutton 1009. After completing the setting, the user can return to thedevice identification screen 801 of FIG. 8 by swiping the setting screen1001, for example.

Referring back to FIG. 7, the description of the sequence chart iscontinued.

At step S709 of FIG. 7, upon detecting that the execution operation forexecuting image processing is input to the information terminal 102, theinformation terminal 102 transmits the specifying information to therelay server 101 (step S710).

FIG. 11 is a diagram illustrating the specification informationaccording to the embodiment. In the specification information 1101, theinformation on the input device and the information on the output deviceare specified.

In FIG. 11, the information on the input device (input) includesinformation items, such as the type of the image processing apparatus(type), the destination information (host), the information on theavailable data type (mime_type), and the setting information (settings).Similarly, the information on the output device (output) includesinformation items, such as the type of the image processing apparatus(type), the destination information (host), the information on theavailable data type (mime_type), and the setting information (settings).

In the example of FIG. 11, the electronic whiteboard 104-1 (IP address:192.168.1.11; available data type: JPEG, TIFF; setting: none) isspecified as the input device in the specifying information 1101. Inaddition, the MFP 104-2 (IP address: 192.168.1.12; available data type:PDF, Postscript; setting: color, 200 dpi, single-side printing, 2 in 1,two copies) is specified as the output device in the specifyinginformation 1101.

Referring back to FIG. 7, the description of the sequence chart iscontinued.

At Step S711, the data type determination unit 603 of the relay server101 determined the data type of the input data based on the informationon the data type that can be processed in the image processing apparatus104 for inputting, and the information on the data type that can beprocessed in the image processing apparatus 104 for outputting, whichare included in the specifying information.

At step S712, the input data retrieval unit 604 of the relay server 101requests to obtain the input data in the data type determined by thedata type determination unit 603 from the input device (the electronicwhiteboard 104-1).

At step S713, the input data retrieval unit 604 of the relay server 101obtains input data from the input device, and the input data retrievalunit 604 stores the obtained input data in the storage unit 607 as thecontent data 609, for example.

At step S714, depending on necessity, the converter 605 of the relayserver 101 converts the content data 609 obtained by the input dataretrieval unit 604 into data in the data type that can be processed bythe output device (the MFP 104-2).

FIG. 12 is a diagram illustrating an example of a convertible listaccording to the embodiment. In the convertible list 1201, informationon the data types that can be converted by the convertor 605 isdescribed. In the example of FIG. 12, it is indicated that the convertor605 can support conversion from JPEG to PFD, conversion from MSWORD toPDF, and conversion from PDF to JPEG.

At step S715 of FIG. 7, the output data transmitter 606 of the relayserver 101 transmits a request for outputting output data to the outputdevice (the MFP 104-2), and subsequently the output data transmitter 606transmits the output data to the output device at step S716.

At step S717, the relay server 101 transmits, to the informationterminal 102, a completion notice indicating that the image processingis completed. At step S718, the information terminal 102 that receivesthe completion notice causes the display input unit 617 to display acompletion screen.

By the above-described process, the output data that is based on theinput data obtained from the input device (the electronic whiteboard104-1) can be output to the output device (the MFP 104-2). Additionally,at this time, the input data and the output data are not stored in theinformation terminal 102, so that the input data and the output data canbe prevented from being leaked.

Here, the determination process and the conversion process of steps fromS711 to S714 of FIG. 7 are described in detail.

(Determination Process and Conversion Process)

FIG. 13 is a flowchart illustrating the determination process and theconversion process of the data type according to the embodiment.

At step S1301, the relay server 101 determines whether the mime_type ofthe input device (the image processing apparatus 104 for inputting) andthe mime_type of the output device (the image processing apparatus 104for outputting) are the same, based on the specifying informationreceived from the information terminal 102, for example. In this manner,the relay server 101 determines whether there is a data type that can beprocessed by both the input device and the output device.

In response to detecting, at step S1301, that there is a mime_type incommon with the input device and the output device, the relay server 101obtains the input data in the detected mime_type from the input device(step S1302). For example, in response to detecting that the inputdevice and the output device support the same mime_type “JPEG,” therelay server 101 transmits, to the input device, a request 1401 forobtaining the input data, such as the request depicted in FIG. 14A. Inthe example of FIG. 14A, the request 1401 for obtaining the input dataincludes information specifying the JPEG format, as the data type of theinput data.

The input device that receives the request for obtaining the input dataobtains image data in the JPEG format, which is the specified mime_type;and the input device transmits the obtained image data to the relayserver 101. Further, the relay server 101 causes the output device tooutput the obtained data in the JPEG format (step S1303), for example.Namely, in response to detecting that there is a data type that can beprocessed by both the input device and the output device, the relayserver 101 causes the output device to output the data withoutconverting the data type.

Whereas, in response to detecting, at step S1301, that there is nomime_type that is in common with the input device and the output device,the relay server 101 determines whether there is a combination of themime_type of the input device and the mime_type of the output devicesuch that the mime_type of the input device can be converted into themime_type of the output device (step S1304). The relay server 101obtains the convertible list that is illustrated in FIG. 12 from theconverter 605, and the relay server 101 determines whether there is acombination of the mime_type of the input device and the mime_type ofthe output device such that the mime_type of the input device can beconverted into the mime_type of the output device.

In response to detecting, at step S1304, that there is no combination ofthe mime_type of the input device and the mime_type of the output devicesuch that the mime_type of the input device can be converted into themime_type of the output device, the process proceeds to step S1305. Atstep S1305, the relay server 101 repots the conversion error to theinformation terminal 102 (step S305), and the process ends.

Whereas, in response to detecting, at step S1304, that there acombination of the mime_type of the input device and the mime_type ofthe output device such that the mime_type of the input device can beconverted into the mime_type of the output device, the process fromsteps S1306 to S1308 is executed.

At step S1306, the relay server 101 obtains the data in the convertiblemime_type from the input device.

At step S1307, the relay server 101 converts the obtained data into datain a mime_type that can be processed by the output device.

At step S1308, the relay server 101 causes the output device to outputthe converted data in the mime_type that can be processed by the outputdevice. For example, after transmitting a request 1402 for outputtingthe output data, such as the request illustrated in FIG. 14B, to theoutput device, the relay server 101 transmits the output data to theoutput device. The example of FIG. 14B indicates that the output devicereceives the output data in the PDF format from the relay server 101,and the output device executes image processing (e.g., printing) withsettings of the color mode, 200 dpi, the single-side printing, 2 in 1,and two copies.

As described above, the relay server 101 according to the embodimentobtains, based on the specifying information that is received from theinformation terminal 102, the input data from the image processingapparatus 104 for inputting, and the relay server 101 causes the imageprocessing apparatus 104 for outputting to output the output data, whichis based on the obtained input data. Thus, the information terminal 102can control image processing by the plurality of image processingapparatuses 104, without storing, in the information terminal 102, theimage data to be processed.

In this manner, the image processing system 100 can be provided thatmakes it easier to control, by using the information terminal 102, suchas a smart device, image processing that is to be executed by theplurality of image processing apparatuses 104-1 to 104-4, whilepreventing information of the data to be processed from being leaked.

[Another Embodiment]

In the above-described embodiment, the execution operation (step S709)is made, after identifying (specifying) the input device and the outputdevice at step S703 to step S708 of FIG. 7, for example. However, theimage processing apparatus 104 for inputting and the image processingapparatus 104 for outputting can be separately specified.

<Functional Configuration>

FIG. 15 is a diagram illustrating an example of the functionalconfiguration of the image processing system 100 according to theembodiment. In FIG. 15, the relay server 101 includes, in addition tothe functional configuration of the above-described embodiment, which isdepicted in FIG. 6, an identifier reporting unit 1501. Here, thedifferences from the configuration of the image processing system 100according to the above-described embodiment are mainly described. Theother configuration of the image processing system 100 according to thisembodiment is the same as the configuration of the image processingsystem 100 according to the above-described embodiment.

In response to detecting that the specifying information for specifyingthe image processing apparatus 104 for inputting is received from theinformation terminal 102, the identifier reporting unit 1501 associatesthe received specifying information with an identifier (ticket) foridentifying the current process, and the identifier reporting unit 1501stores the received specifying information and the associated identifierin the storage unit 607, as input device information 1502. Further, theidentifier reporting unit 1501 reports (transmits) the identifier foridentifying the current process to the information terminal 102 thattransmits the specifying information.

<Flow of the Process>

FIG. 16 is a sequence chart illustrating an example of image processingaccording to the embodiment.

At step S1601, similar to the above-described embodiment, a userperforms an operation to specify the input device (specifyingoperation), in accordance with the device identification screendisplayed on the display input unit 617 of the information terminal 102.

At step S1602, similar to the above-described embodiment, theinformation terminal 102 transmits a request to obtain deviceinformation to the image processing apparatus 104 for inputting that isidentified at step S1601, such as the electronic whiteboard 104-1.

At step S1603, similar to the above-described embodiment, theinformation terminal 102 obtains the device information from the imageprocessing apparatus 104 for inputting to which the request to obtainthe device information is transmitted, such as the electronic whiteboard104-1.

At step S1604, in the embodiment, the user performs the operation tospecify the input device, without performing the operation to identifythe output device. For example, the user performs the operation tospecify the input device by pressing the run button 804 in the deviceidentification screen 801 of FIG. 8, without selecting the outputdevice.

At step S1605, the information terminal 102, which receives theoperation to specify the input device from the user, transmits, to therelay server 101, specifying information 1701 for specifying the inputdevice, such as the specifying information 1701 illustrated in FIG. 17A.Compared with the specifying information 1104 according to theabove-described embodiment, the specifying information 1701 forspecifying the input device does not include the item of “output.”

In response to detecting, at step S1606, that the specifying information1701 for specifying the input device is received from the informationterminal 102, the relay server 101 transmits, to the informationterminal 102, identification information 1702 for identifying theprocess, such as the information illustrated in FIG. 17B. In addition,at this time, the relay server 101 associates the information of theinput device, which is included in the specifying information that isreceived from the information terminal 102 and that is for specifyingthe input device, with the identification information 1702 foridentifying the process, which is transmitted to the informationterminal 102. Then, the relay server 101 stores, in the storage unit607, the information of the input device and the associatedidentification information 1702 for identifying the process, as theinput device information 1502. At this time, the relay server 101 maynot obtain the input data from the input device. Note that theidentification reporting unit 1501 creates, for each process, a uniquevalue, as the value of the “ticket” included in the identificationinformation 1702 of this process, for example.

At step S1607, the user performs the operation to identify the outputdevice, in accordance with the device identification screen displayed onthe display input unit 617 of the information terminal 102. Here, theoperation to identify the output device may be performed, for example,immediately after the operation to specify the input device, asillustrated in FIG. 16. Alternatively, the operation to identify theoutput device may be performed later.

At step S1608, the information terminal 102 transmits a request toobtain the device information to the output device that is identified atstep S1607, such as the MFP 104-2.

At step S1609, the information terminal 102 obtains the deviceinformation from the output device, such as the MFP 104-2, to which therequest to obtain the device information is transmitted.

In response to detecting, at step S1610, that the operation to executeimage processing is received from the user, the image processing system100 executes image processing based on the received specifyinginformation (steps S710 to S718), similar to the above-describedembodiment.

FIG. 18 illustrates an example of the specifying information 1801 thatis transmitted from the information terminal 102 to the relay server 101at this time. The specifying information 1801 according to theembodiment includes the item “ticket,” instead of the item “input” inthe specifying information 1101, which is illustrated in FIG. 11, of theabove-described embodiment. The item “ticket” is included in theidentification information 1702 for identifying the process, and theidentification information 1702 is reported from the relay server 101 atstep S1606.

The relay server 101 can identify the device information of the inputdevice, based on the item “ticket” included in the specifyinginformation 1801 and the input device information 1502 stored in thestorage unit 607.

By the above-described process, the user can specify the input deviceand the output device at different timings, thereby enhancing theconvenience of the image processing system 100.

[Further Embodiment]

In this embodiment, an example of a case is described where at least oneimage processing apparatus 104 is preregistered in the relay server 101.

<Functional Configuration>

FIG. 19 is a diagram showing an example of the functional configurationof the image processing system 100 according to this embodiment.

(Functional Configuration of the Relay Server)

The relay server 101 according to the embodiment includes, in additionto the configuration of the relay server 101 according to theabove-described embodiment, which is illustrated in FIG. 6, a deviceinformation retrieval unit 1901; and a list information transmitter1902. Here, the differences from the configuration of the imageprocessing system 100 according to the above-described embodiment aremainly described because the other configuration of the image processingsystem 100 according to this embodiment is the same as the configurationof the image processing system 100 according to the above-describedembodiment.

The device information retrieval unit 1901 obtains, from the inputdevice, the device information of the input device; and the deviceinformation retrieval unit 1901 obtains, from the output device, thedevice information of the output device. Further, the device informationretrieval unit 1901 stores, for example, the obtained device informationin the storage unit 607, as device information 1903.

In response to receiving a request from the information terminal 102,for example, the list information transmitter 1902 transmits listinformation of the at least one image processing apparatus 104, which isstored as the device information 1903, to the information terminal 102.

Here, the device information retrieval unit 1901 and the listinformation transmitter 1902 can be implemented, for example, by one ormore programs executed by the CPU 301 of FIG. 3.

(Functional configuration of the information terminal) The informationterminal 102 according to the embodiment includes, in addition to theconfiguration of the information terminal 102 according to theabove-described embodiment, which is illustrated in FIG. 6, aregistration request unit 1904; and a list information request unit1905. The information terminal 102 according to the embodiment may notinclude the device information retrieval unit 612, which is included inthe information terminal 102 according to the above-describedembodiment. Here, the differences from the configuration of the imageprocessing system 100 according to the above-described embodiment aremainly described because the other configuration of the image processingsystem 100 according to this embodiment is the same as the configurationof the image processing system 100 according to the above-describedembodiment.

The registration request unit 1904 is a unit for requesting, from therelay server 101, a registration process for registering the imageprocessing apparatus 104. The registration process for registering theimage processing apparatus 104 is described below.

The list information request unit 1905 is a unit for requesting, fromthe relay server 101, the list information of the at least one imageprocessing apparatus 104.

The registration request unit 1904 and the list information request unit1905 can be implemented, for example, by one or more programs executedby the CPU 401.

<Flow of the Process>

(Registration Process)

FIG. 20 is a sequence chart illustrating an example of the registrationprocess according to the embodiment.

At step S2001, the user performs an operation to register the imageprocessing apparatus 104 that is desired to be registered, such as theelectronic whiteboard 104-1. The operation for the registration may beperformed, for example, by adding a selection button for selecting adevice to be registered in the device identification screen 801 depictedin FIG. 8, and by pressing the selection button for selecting the deviceto be registered. In response to receiving, at step S2002, the operationfor registering, for example, the electronic whiteboard 104-1 from theuser, the information terminal 102 transmits, to the relay server 101, adevice registration request for registering the electronic whiteboard104-1 by using the registration request unit 1904.

In response to receiving, at step S2003, the device registration requestform the information terminal 102, the relay server 101 transmits arequest to obtain device information to the electronic whiteboard 104-1by using the device information retrieval unit 1901.

At step S2004, the device information retrieval unit 1901 of the relayserver 101 obtains the device information transmitted from theelectronic whiteboard 104-1.

At step S2005, the device information retrieval unit 1901 of the relayserver 101 registers the device information obtained from the electronicwhiteboard 104-1 in the device information 1903 of the storage unit 607.At this time, the device information retrieval unit 1901 generates aunique device ID (e.g., “whiteboard 1”) for the electronic whiteboard104-1. Then, the device information retrieval unit 1901 associates thegenerated device ID with the obtained device information, and the deviceinformation retrieval unit 1901 registers the generated device ID andthe associated device information in the device information 1903.

At step S2006, the relay server 101 reports to the information terminal102 that the registration process is completed.

In response to receiving the completion notice at step S2007, theinformation terminal 102 displays, on the display input unit 617,information indicating that the registration process is completed.

Similarly, the user can register a plurality of image processingapparatuses 104.

For example, at step S2008, the user performs an operation to registerthe information processing apparatus 104, such as the MFP 104-2, whichis desired to be registered, subsequently.

In response to receiving, at step S2009, the operation to register, forexample, the MFP 104-2 from the user, the information terminal 102transmits a request to register the MFP 104-2 to the relay server 101 byusing the registration request unit 1904.

Upon receiving, at step S2010, the device registration request from theinformation terminal 102, the relay server 101 transmits a request toobtain device information to the MFP 104-2 by using the deviceinformation retrieval unit 1901.

At step S2011, the device information retrieval unit 1901 of the relayserver 101 obtains the device information transmitted from the MFP104-2.

At step S2012, the device information retrieval unit 1901 of the relayserver 101 registers the device information obtained from the MFP 104-2in the device information 1903 in the storage unit 607.

At this time, the device information retrieval unit 1901 generates aunique device ID (e.g., “mfp 1”) for the MFP 104-2. Then, the deviceinformation retrieval unit 1901 associates the generated device ID withthe obtained device information, and the device information retrievalunit 1901 registers the generated device ID and the associated deviceinformation in the device information 1903.

At step S2013, the relay server 101 reports to the information terminal102 that the registration process is completed.

Upon receiving, at step S2007, the completion notice, the informationterminal 102 displays, on the display input unit 617, informationindicating that the registration process is completed.

(Image Processing)

FIG. 21 is a diagram illustrating an example of the image processingaccording to the embodiment.

At step S2101, a user performs, on the information terminal 102, anoperation to display a list of available devices.

At step S2102, the list information request unit 1095 of the informationterminal 102, which receives the operation to display the list of thedevices from the user, transmits, to the relay server 101, a request toobtain list information.

At step S2103, the list information transmitter 1902 of the relay server101, which receives the request to obtain the list information,transmits, to the information terminal 102, the list information of theat least one available image processing apparatus 104. Then, theinformation terminal 102 obtains the transmitted list information. FIG.22 illustrates an example of the list information transmitted by therelay server 101 at this time. The list information 2201 includes, forexample, the device information of the image processing apparatuses 104that is registered at steps S2205 and S2012 of FIG. 20; and the deviceIDs that are unique to the corresponding image processing apparatuses104.

In the example of FIG. 22, the list information 2201 includes the deviceinformation of the MFP 104-2, and the device ID “mfp 1” corresponding tothe MFP 104-2. In addition, the list information 2201 includes thedevice information of the electronic whiteboard 104-1, and the device ID“whiteboard 1” corresponding to the electronic whiteboard 104-1.

Referring back to FIG. 21, the description of the sequence chart iscontinued.

At step S2104, the information terminal 102 causes the display inputunit 617 to display a selection screen for selecting an input deviceand/or an output device. The selection screen displays a list of thedevice ID of the at least one available image processing apparatus 104and an icon of the device ID on a graphical user interface (GUI). FIG.23 illustrates an example of the selection screen for selecting an inputdevice and/or an output device, which is displayed on the display inputunit 617, at this time.

In the example of FIG. 23, the selection screen 2301 for selecting aninput device and/or an output device is displayed on the display inputunit 617 of the information terminal 102. The selection screen 2301includes the icon and the device ID “whiteboard 1” of the electronicwhiteboard 104-1; and the icon and the device ID “mfp 1” of the MFP104-2. Additionally, the selection screen 2301 includes an icon and adevice ID of another image processing apparatus 104 included in the listinformation 2201, such as the icon and the device ID “projector 1” ofthe projector 104-4.

At step S2105 of FIG. 21, the user performs an operation to select aninput device and/or an output device in the selection screen 2301, whichis displayed on the display input unit 617 of the information terminal102. If a user desires to output, from the MFP 104-2, image data that isinput from the electronic whiteboard 104-1, for example, the user maydrag and drop the icon 2302 of the electronic whiteboard 104-1 onto theicon 2303 of the MFP 104-2. According to the embodiment, an input deviceand an output device can be easily selected, in such a manner.

Upon receiving, at step S2105, the operation to execute image processingfrom the user, the image processing system 100 executes image processingbased on the received specifying information (steps S710 to S718),similar to the above-described embodiment.

FIG. 24 illustrates an example of the specifying information that istransmitted from the information terminal 102 to the relay server 101 atthis time. Instead of the destination information and the deviceinformation of the input device in the specifying information 1101according to the above-described embodiment, which is illustrated inFIG. 11, the specifying information 2401 according to the embodimentincludes “whiteboard 1,” which is the device ID of the input device (theidentification information of the image processing apparatus 104 forinputting). Furthermore, instead of the destination information and thedevice information of the output device in the specifying information1101 according to the above-described embodiment, which is shown in FIG.11, the specifying information 2401 includes “mfp 1,” which is thedevice ID of the output device (the identification information of theimage processing apparatus 104 for outputting).

According to the embodiment, an input device and an output device can beeasily selected because a list of the at least one available imageprocessing apparatus 104 is displayed on the display input unit 617 ofthe information terminal 102.

An image processing system 100 according to the embodiment of thepresent invention includes at least one image processing apparatus 104;an information terminal 102; and an information processing device 101configured to communicate with the at least one image processingapparatus 104 and the information terminal 102.

The information terminal (102) includes a specifying informationtransmitter 613 configured to transmit, to the information processingdevice 101, specifying information 1101 for specifying, from the atleast one image processing apparatus 104, an image processing apparatusfor inputting and an image processing apparatus for outputting.

The information processing device 101 includes an input data retrievalunit 604 configured to obtain, based on the specifying information 1101received from the information terminal 102, input data from the imageprocessing apparatus for inputting. The information processing device101 further includes an output data transmitter 606 configured totransmit, based on the received specifying information 1101, output databased on the obtained input data to the image processing apparatus foroutputting.

By the above-described configuration, the image processing system 100may make it easier to control, by using the information terminal 102,such as a smart device, image processing that is to be executed by theplurality of image processing apparatuses 104, while preventinginformation of the data to be processed from being leaked.

The information processing system, the image processing method, and thenon-transitory storage medium storing the image processing program aredescribed by the above-described embodiments. However, the informationprocessing system, the image processing method, and the non-transitorystorage medium storing the image processing program according to thepresent invention are not limited to the above-described embodiments,and various modifications and improvements may be made within the scopeof the present invention. Specific examples of numerical values are usedin order to facilitate understanding of the invention. However, thesenumerical values are simply illustrative, and any other appropriatevalues may be used, except as indicated otherwise. The separations ofthe items in the above description are not essential to the presentinvention. Depending on necessity, subject matter described in two ormore items may be combined and used, and subject matter described in anitem may be applied to subject matter described in another item(provided that they do not contradict). A boundary of a functional unitor a processing unit in a functional block does not necessarilycorrespond to a boundary of a physical component. An operation by aplurality of functional units may be physically executed by a singlecomponent. Alternatively, an operation by a single functional unit maybe physically executed by a plurality of components. For the convenienceof explanation, the devices according to the embodiment of the presentinvention are explained by using the functional block diagrams. However,these devices may be implemented in hardware, software, or combinationsthereof. The software that operates in accordance with the presentinvention may be prepared in any appropriate storage medium, such as arandom access memory (RAM), a flash memory, a read-only memory (ROM), anEPROM, an EEPROM, a register, a hard disk drive (HDD), a removable disk,a CD-ROM, a database, a server, and so forth.

The present invention can be implemented in any convenient form, forexample using dedicated hardware, or a mixture of dedicated hardware andsoftware. The present invention may be implemented as computer softwareimplemented by one or more network processing apparatuses. The networkcan comprise any conventional terrestrial or wireless communicationsnetwork, such as the Internet. The processing apparatuses can compromiseany suitable programmed apparatuses such as a general-purpose computer,personal digital assistant, mobile telephone (such as a WAP or3G-compliant phone) and so on. Since the present invention can beimplemented as software, each and every aspect of the present inventionthus encompasses computer software implementable on a programmabledevice. The computer software can be provided to the programmable deviceusing any storage medium for storing processor readable code such as afloppy disk, a hard disk, a CD ROM, a magnetic tape device or a solidstate memory device. The hardware platform includes any desired hardwareresources including, for example, a central processing unit (CPU), arandom access memory (RAM), and a hard disk drive (HDD). The CPU mayinclude processors of any desired kinds and numbers. The RAM may includeany desired volatile or nonvolatile memories. The HDD may include anydesired nonvolatile memories capable of recording a large amount ofdata. The hardware resources may further include an input device, anoutput device, and a network device in accordance with the type of theapparatus. The HDD may be provided external to the apparatus as long asthe HDD is accessible from the apparatus. In this case, the CPU, forexample, the cache memory of the CPU, and the RAM may operate as aphysical memory or a primary memory of the apparatus, while the HDD mayoperate as a secondary memory of the apparatus.

What is claimed is:
 1. An information terminal comprising: a networkinterface that connects, through a network, to an information processingdevice for requesting an output device to execute an output processdepending on a processing result of an input device for executing aninput process; a processor; and a memory that includes instructions,which when executed, cause the processor to execute the following steps:receiving, from one device among the input device and the output device,capability information including a first setting corresponding to afirst process executable by the one device and a second settingcorresponding to a second process executable by the one device, andconnection information for connecting to the one device, the processorreceiving the first setting and the second setting from the same device;receiving an input via a screen displayed on a display; the inputindicating that the one device is the input device or the output device;determining the first setting or the second setting, based on whetherthe input indicates that the one device is the input device or theoutput device; requesting the information processing device to causeexecution of the input process and the output process, based on thedetermined first setting or second setting, and the connectioninformation; wherein the executed input process or the executed outputprocess is the first process or the second process corresponding to thedetermined first setting or second setting, and is executed by the onedevice.
 2. A method executed by an information terminal including anetwork interface that connects, through a network, to an informationprocessing device for requesting an output device to execute an outputprocess depending on a processing result of an input device forexecuting an input process, the method comprising: receiving, by theinformation terminal from one device among the input device and theoutput device, capability information including a first settingcorresponding to a first process executable by the one device and asecond setting corresponding to a second process executable by the onedevice, and connection information for connecting to the one device, theinformation terminal receiving the first setting and the second settingfrom the same device; receiving, by the information terminal, an inputvia a screen displayed on a display; the input indicating that the onedevice is the input device or the output device; determining the firstsetting or the second setting, based on whether the input indicates thatthe one device is the input device or the output device; requesting, bythe information terminal, the information processing device to causeexecution of the input process and the output process, based on thedetermined first setting or second setting, and the connectioninformation; wherein the executed input process or the executed outputprocess is the first process or the second process corresponding to thedetermined first setting or second setting, and is executed by the onedevice.
 3. A system comprising: an information terminal; an informationprocessing device that requests an output device to execute an outputprocess depending on a processing result of an input device, wherein theinformation terminal includes a network interface that connects to theinformation processing device through a network; a processor; and amemory that includes instructions, which when executed, cause theprocessor to execute the following steps: receiving, from one deviceamong the input device and the output device, capability informationincluding a first setting corresponding to a first process executable bythe one device and a second setting corresponding to a second processexecutable by the one device, and connection information for connectingto the one device, the processor receiving the first setting and thesecond setting from the same device; receiving an input via a screendisplayed on a display; the input indicating that the one device is theinput device or the output device; determining the first setting or thesecond setting, based on whether the input indicates that the one deviceis the input device or the output device; requesting the informationprocessing device to cause execution of the input process and the outputprocess, based on the determined first setting or second setting, andthe connection information; wherein the executed input process or theexecuted output process is the first process or the second processcorresponding to the determined first setting or second setting, and isexecuted by the one device.